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ABSTBACT 

A mathematical model for computer-aided instruction 
has been developed. The assumption is made that the course is divided 
into a hierarchy of levels of difficulty and that if a student is 
able to perform successfully at a given level qi difficulty, he can 
also perform successfully at all levels of lesser difficulty. 
Furthermore, if a student performs successfully at one level, the 
probability of successful performance at the next higher level is ^ \^ 
increased. Given the initial vector of probabilities for successful 
performance at each level, and the expected times it takes to attempt 
a successful performance at each level, the model computes an 
instructional sequence vhich minimizes the expected time required for 
the student to complete the course by performing successfully at the 
highest level of difficulty. Dynamic programing is used to find this 
sequence. (Author) 
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ABSTRACT 



A mathematical model for computer aided instruction is ^ 
developed. It is assumed that the course is divided into a hierarchy 
of levels of difficulty. These levels are such that if a student 
is able to perform successfully at a given level of difficulty, he 
can also perform successfully at all levels of lesser difficulty. 
Furthermore, if a student performs successfully at one level, 
it increases his probability of being able to perform successfully 
at the next higher level of difficulty. Given the initial vector 
of probabilities for successful performance at each level, the 
vector describing how l^ese probabilities change with successful 
performances at each level, and the expected times it takes to 
attempt a successful performance at each level, this model 
computes an instructional sequence that minimizes the expected 
time required for the student to complete the course by performing 
successfully at the highest level of difficulty. Dynamic programming 
is used to find this sequence. 
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_A MARKOV DECISION MODEL FOR COMPUTED - 
AIDED INSTRUCTION 

I. INTRODUCTION 

Several researchers have been interested in the application 
of optimization techniques to models of learning and instruction. 
Karush and Dear (1966) developed an optimal strategy for teaching 
students to learn a list of independent items. The basic assumption 
was that an item is either in a learned or unlearned state. If It is 
given in the unlearned state it goes to the learned state with 
probability c, while once it reaches the learned state it stays 
there. Atkinson and Paulson (1972) described experiments in 
which extensions of this model were applied to computer-assisted 
spelling instruction with elementary school children. Chant and 
Atkinson (1973) developed an optimization technique for allocating 
instructional effort to two interrelated blocks or strands of 
learning material. Their key assumption was that the learning 
rate for each of the two strands depends solely on the difference 
between the achievement levels on the two strands. 

The model of this report concerns a system where a student 
is to be taught to perform a certain skill at a given level of 
competence. He achieves this by working problems through or 
taking tests at various levels of difficulty. It is assumed that 
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if a student is able to perform successfully at one level of difficulty 
he is able to perform at the next lower or preceding level of difficulty 
and consequently at all lower levels of difficulty. This assumption 
is particularly applicable in the following two situations. 

The first situation is one where the material covered at one 
level includes all that covered at preceding levels, plus some 
additional material. An example of this is a program developed 
at Behavioral Technology I^aboratories to teach students Kirchoff's 
laws. This course is comprised of eleven levels with the lowest 
level defining the units for voltage, current and resistance up to 
the highest level which deals v*ith the application of Ohm's law 
and Kirchoffs voltage and current laws in complex networks. 

The second situation is one wnere the material and problems 
covered at a particular level are virtually the same as at the 
immediately preceding level except more clues and hints are given 
at the preceding level, A good example of this would be a version 
of the Kirchoff's laws program considered earlier at Behavioral 
Technology Laboratories in which problems would be given at the 
following levels: 

1. Problems are given in steps with cues and knowledge 
of results at each step. 

2. Problems are given in steps with no cues or knowledge 
of results at each step. 

3. The student solves problems in steps but he chooses the steps. 
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4, The student is simply given problems and asked to solve 
them. 

Note, however, the assumption given for this model would not 
be applicable for the situation where a given level did not use certain 
material introduced at preceding levels. 

It is also assumed that if a student performs successfully at 
one level, it will increase his probability of being able to perform 
successfully at the next higher level. The student completes the 
course when he performs successfully at the highest level. The aim 
of the model presented in this paper is to choose the levels at which 
problems should be assigned in ^^e course sequence so the expected 
time required by the student to complete the course is minimized. 

n. THE MODEL 

Mathematical Formulation 

The problem of instructing the student so that he completes the 
course in minimum time is formulated as a Markov decision process. 
The set of actions are 1, . . . , N where action i is that of giving the 
student a problem at level i. The levels are numbered in decreasing 
order of difficulty. Thus level 1 is the hardest and level N the 
easiest. The state 0 is that in which the student has performed 
successfully at level 1. The states in which the student has not 
performed successfully at level 1 are characterized by the vectors 
P " • • • • Pjj) where p^ equals the probability that the 
student will correctly do a problem at level i. It is assumed 



that if a student can do a problem at level i, he can also do it 
at level j for all j > i. Thus, is non-decreasing in i. 
For each action i, let 

= P [student can perform at level i - l/istudent completes 
problem at level i correctly and could not perform 
at level i - 1 before] . 
Thus, if the state is p and we perform action 1, we go to 
state 0 with probability pj^ and remain in state p with probability 
(1 - pj). If we take action i > 1 we go to state p with probability p. 

where p^j Pi-l ^i^^ * Pi Pfe ~ Pfc' k ^ i - If and remain in state 

p with probability (1 - Pj). 

Equivalently, the components of p above may be represented 
by the following: 

Pi.l = (1 - qj) Pi_i + 

(1) 

pj^ = Pj^, k ^ i-1 . 

Once the system reaches state 0, it remains there. Associated with 
each action, i, is a cost c. which may be equal to the expected time 
it takes to attempt a problem at level i. It is desired to choose an 
action policy that reaches state 0 at minimum cost. 

Some Solution Properties 

A policy specifies an action for each state of the system other 
than state 0. Let V(ir, p) be the total expected cost under policy ir 
when the system is in state p. If i is the action specified for state p 
by IT, then it follows that: 
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V(ir,p) = c. + p.V(Tr.p) + (1 . p.)V(Tr,p) 

(2) 

where p^^j = (1-q^) p^^j + and pj^ = pj^ for k ^ i-l. 

It is of course desired to find ir so that V(ir, p) < V(7f, p) 
for all p and all ir« 

Note that if action i is taken and the student does not complete 
his task at level i successfully, the state does not change and action 
i will be repeated. Thus any action taken will be repeated until the 
student completes a problem correctly at which time the state changes 
and a new action may be taken. In addition, the state resulting from 
the first time the student performs correctly at level i is independent 
of the number of attempts it takes the student to perform successfully 
at that level. Thus ir and p determine a sequence of correct 
responses at each level though not the number of trials necessary 
to obtain these responses. Of course, the sequence must end with 
one correct response at level 1. The expected niunber of trials 
necessary for the student to complete a problem successfully at 
level i is l/p. and the expected cost of this is c^/p.^. 

Consider the policy that requires performance at level 1 only 
for p. The cost of this policy would be c^^/p^^ which would be less 
than that of any policy requiring more than c^/c^p^ successful, 
performances at level i. Hence, the set of performance sequences 
for p that are superior to testing at level I only is finite and there 
is an optimal sequence for each p. This establishes the existence of 
a policy IT satisfying V(7r,p) < V(if,p) all p and all if. 
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We are now ready to prove the following theorems. 

Theorem 1: If p - r >0, then V(Tr,p) < V(Tr,r), if ir is an 
optimal policy. 

Proof: If tr consists of one correct response (at level 1) for 
both p and r, then V(ir,p) = c^/pj while V(ir, r) = r^ and the 
theorem holds. Suppose the theorem holds for all p and r such 
that IT specifies n or fewer total correct responses for r. 
Consider p and r such that ir requires n + 1 or fewer correct 
responses for r and let i be the level at which the first correct 
response for state r must take place. Then V(ir, p) < ^j/Pi P) 
and V(ir, r) = c./r, + V(Tr,f) where from (1) Pj j = 

(1 - qj) Pj.i + r.^j = (1 - q.) r.^j Pfc Pfc 'k ^ ^ 

Thus p > r, f requires n or fewer correct responses and 
V(ir, p) < V(Tr, r). The theorem follows from induction. 

TJlieorem 2 ; There is an optimal policy ir such that if 
^$ • • • t is the sequence of the levels of correct responses for 
state p, then a^^ > all k. 

Proof: Let ir be an optimal policy and a^, b^, . . . , a^ be the 
sequence specified for p. Suppose there is a k such that < a^^^^ . 

Let p be the state and c the expected cost resulting from the correct 
responses to the sequence a^i ^2^, . , initial state p. 

Consider the policy tt which differs from ir only in that a^^ and 
are interchanged in the sequence for p and let i = j = 



If j < i-1. then V(ir, p),= V(if, p) = c + c^/p^ + c./p. + V(Tr, r) where from (1), 



consequently V(Tr, p) = V(if, p). Also from (1), if j = i-1, then 



V(ir, p) = c + Cj/pj + Cj^j/pj^j + V(Tr, r), where r is defined above while 



V(ff,p) = 5+c./p. +c.y [(l-q.)p.^j+q.] +V(7r,r), Thus V(^, p) < V(ir, p) 



and if is also optimal. Continuing in this manner, an optimal sequence for 
p is eventually obtained in which the members of the sequence are in 
non-incr easing order. The theorem follows since p is general. 

Thus die search for an optimal policy may be confined to those 
which yield a sequence of correct responses at levels that are non-increasing 
in the sequence. 

As noted before, if one elicits one correct response at level i, 
p. J is transformed to (l-q^) Pj^j + q^. Applying this transformation 

recursively it follows that if one elicits k correct responses at level 

k k-1 

i, pj^j is transformed to (l-q^) pj^j + (l"<Ji) q^ + •••+ q^ which 
k 

sums to 1 - (l-q^) (1-pj^j). Thus, if ir is a policy of the above 
type and specifies k(i) correct responses at level i for p, then 



N 

V(ir, p) = E k(i) c./p. where 
i=l ^ ^ 



(3) 



p, = l-(l-p.) (l.q.^j) 



>(i+l) 



for i<N 




7- 



ERLC 



Let V^(p) be the minimum cost for state p if we restrict 



instruction to levels 1, . . . ,n. That is no instruction takes place 
at levels n + 1, n + 2, . . . , N. Of course, only the first n 
components of p are relevauit in determining V^(p) and throughout 

the remainder of this paper it will be assumed that p is restricted 
to Pjt • • • t Pn in ^n^PV* 

In other parts of this paper, the symbol p will be used to 
represent restrictions of p to certain components where the 
restriction is obvious. In particular, in the expression/V^(p, p^), 
p represents the restriction of p to p^, P2''"'Pn-r 

From Theorem 2 and (3) if follows that 




min 
k 




where 



(4) 




Vi(p) 




Note that in the right hand side of the second line of (4) p 



represents the restriction of p to 



Algorithm for Two Levels 

For the two levels problem it follows from (4) that 




where 



(5) 



V^(p) = k C2/p^ + c^/ (1 - (l-q2)^ (1-p^)) 
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Consequently, 

^(P) - V^"^P) = C2/P2 + Cj/(1 -(l-q2)Nl-Pi)) - Cj/(l.(l-q2)^"^(l-Pi)), (6) 

V^(p) < V^~^(p)if and only if the expression in (6) is less than or 
equal to zero. This happens if and only if 



P|2 > f(k» Pj) where 



f(k, pj) = 



for k = 1, 2 . . . 



(l-qg)^ (l-Pj) - (Z-qg) + ^ 



(l-qg) (1-Pi) _ 



Cjqg (7) 



This, requiring a student to perform successfully k times at 
level 2 is preferred to requiring him to perform successfully k-1 
times at level 2 if r> ^ f(k, p^), while requiring him to perform 

successfully k-1 time at level 2 is preferred if p^ < f(kip^)i and 

these two strategies yield equal costs if = ^(^^9 P^)* 

Theorem 3 ; In (7)i f(k, p^) is nondecreasing in k and is 
non-negative in all k. 

Proof; Substituting in (7) and rearranging terms one obtains 



non- 



f(l,Pj) = Cg j^Pjqg + (1- (l-Pj)) / (1-Pi) j /cj qg which is clearly 
negative. Also, f(k, p^) - f(k.l, pj)=C2[^l/(l-q2)^"^l-Pl) -(l-qg)^*^^ /c^ 

which is non-negative since the positive term of the second factor in 
the numerator exceeds 1 while the negative one is less than 1. 
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Thus, f(kip^) is increasing in Q.E.D, 

Theorem 4; Define f(0,pj) =0 and f (k, pj^) as in (7) for 

k = 1,2, • • • I m, where m is such that f(m, pj^) > 1 and f(m-l, p^) < 1. 

Then the value of k that minimizes (5) is that which satisfies 
f(k,pj) < p^ < f(k +l,pj). 

Proof: If i<k, it follows immediately from Theorem 3 that 

V^(p) < V^'-^Cp) < • • • < (p) and similarly if i > k. V^(p) < V^"^^ (p) 

<•••< V^(p). 

Thus from Theorem 4, for fixed p^, the number of successful 
performances required at level 2 to minimize cost is an increasing 
step function of p^, starting at 0 for = 0 and advancing in 

increments of one. The minimizing cost maybe found from (5) once 
k is known. 

Additional Solution Properties 

In calculating ^^(p)^ is much more difficult to get a closed 

form such as that for V^(p) and V^Cp). However, as this section 

will show, for fixed P^^^ji P^^ 2 » • • • »Pi» value of k that minimizes 

(4) is a non -deer easing step function inp with a value of 0 at p =0. 

n * n 

However, the increments of the step function are not necessarily one. 

The next lemma and two theorems show this. First, define 



£„(j,k,p) = (k-j) cj [v^.i(p,l-(lV 'l-P„-l''-\.l(P.l-»-V''<l-Pnll] 
for j < k, p = (pg, . . . , p^) . 

Lemma 5; In (8), f^^J'^' ^ ° defined and V^(p) > vjj(p) 

for p^<f^ (j.k.p); vJ(p) < V^(p) for p^ > f^ (j.k, p); and equality 
holds for p^ = f^(j,k,p). 

Proof; The denominator of f^(j|k, p) is non-negative by- 
Theorem \. The theorem then follows from the definition of 
Vjj(p) in (4). 

Theorem 6; The number of successful performances required 
at level n in order to realize ^^(p) non-decreasing in p^. 

Proof; For i = 1 the theorem obviously holds since only one 
successful performance is required for all p^. For n >1 assume 
the theorem is false. Then there is a system with k > 1, Pj^ ^ P^^ 

such that V^(p,p^) <vj(p,p^) and vj(p,pj< vj;(p,pj. Let 

f (jflc. p) be as defined in Lemma 5, T^ien p <f (jfk, p) and 

n** *^ nn 

p^ > f^(j,k,p) for a contradiction. 

Thus, it has now been shown that for fixed P^f P2' • * * • P^^ 

value of k that minimizes ^^(p) ^ non-decreasing step function 

of p . 

*^n 

Of course, for p^ = 1, the minimizing value of k cannot exceed 
j^(p)/Cj^, since any value exceeding this would be inferior to k = 0. 
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Also. f„(j,k,p) > cj I ^(p,l- (1-Pn-1» " * 

Thus, if V^(p,l-(l-qJ^ (1-P„_i)) < Cn"*" Vl^P'^>' ^J'^'P)^ ^. 
and j successful performances is preferred to k successful 
performances for all k > j. Thus, the sequence of optimal values 
of k in (4) is a subsequence of the set 0, 1, 2,..., min {[V^ l^P^^^n^ 

where j is the smallest integer satisfying 
V ,(p,l- (1-q )j(l-p ,)) < c +V ,(p,l). 

Theorem 7; Suppose j < k < A and f^(j,k, p)> fj^(k,je, p) and 
Pj. P2'***'Pn-l fixed. Then V^(p) \^P^ for any p^. 

Proof; For p^ < f^^U.k.p), vjj[p) < V^(p) while for p^ > f^(j, 
Pn ^ ^n^^' ^ P^ ^i<P> ^ "^^P^* 

Theorem 8: Suppose n(l), n(2), .... n(m) is an increasing 
sequence of integt i'n such that f^(n(i), n(i+l),p) is increasing in 

i and that V^(p) ^ ^n^P'' Pn ^ sequence. 

Then Vjj<'>(p) = V^(p) for f^(n(i-l), n(i), p) < p^ < f^(n(i), n(i+l),p). 

Proof: For p„ > f „ (n(i-l), n (i), p), vj;<^>(p, p^) < >^^'"^>(p, p^) 
<•••< (p.pj for p^< f^(n(i), n(i+l),p), V^^'^P. < 

'^^^(P' Pn) < • • • < ^^""^(P' Pn)- Q. E. D. 
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This means one may start with the sequence 0,1, • • • , min 
{[V i(p)/c ], j } , where j is the smallest integer such that 



that cannot be optimal for any by Theorem 7, continue to eliminate 
from the remaining sequence until the sequence left satisfies the 
conditions of Theorem 8. This procedure must be finite since 
only a finite number of eliminations may occur. 

General Algorithm 

Formally, the algorithm for finding the value of k that 



j(p,l.(l-q^)J(l-p^^^)) < Cj^+V^.i(p)- eliminate those members 



minimizes 




as a function of p_ is as follows: 



Algorithm 1 



1. 



Set n = 1 and V^(p) = c^/p^. 



2. 



If n = N, terminate. Otherwise increase n by one and 



define n(0). n(l) 



I • • • I 



n(m) where n{i) = i and 



m = min {[Vj^_i(p)/c J » j) where j is the smallest 
integer satisfying V^_i(P. 1 - (i-<ln^^(l-Pn-l^^ - 



3. 




4. 



If no i satisfies fj^{n{i), n(i+l).p) > fj^(n(i+l), n(i+2),p) 
delete from the sequence any i such that fj^(n(i), n(i+l), p) > 1 
£iad return to 2 as the value of k which minimizes V^(p) 
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is that which satisfies f^(n(i.l), n(i), p) < p^^ < fj^(n(i), 

n(i+l),p). Otherwise delet«$ any i from the sequence 
satisfying fj^(n(i),n(i+l), p) > f^(n(i+l), n(i+2),p), relabel 

the members of the remaining sequence n(0), n(l), • • • , n(m) 
in increasing order with m 1 equaling the number of 
elements in the remaining sequence and return to step 3, 

Given the sequences generated by algorithm 1, the optimal 
number of successful performances to require at each level and V^(p) 
for n > 1 may be found as follows. 

Algorithm 2 

1. Set m = n and define p = p^, k(n) as the n(j) that 

*^n n 

satisfies f^(n(j-l), n(j), p) < p^^ < fj^(n(j), n(j+l), p). 

2. Decrease m by one. 

k (m"H) 

3. Define P^^^ = 1 - (l^Pm^^^"^m+l^ • "^^^ ^^^^^^ 
k(m) as the n(j) that satisfies f^(n(j-l), n(j), p) < p^ 
< f(n(j), n(j+l),p). 

4. If m=2, define ^ = 1 -(l-Pi)(l-q2)^^^^ and go to 5. 
Otherwise go to 2. 

n 

5. Terminate. V^(p) = £ k(i)c./pj where k(l) = 1. 

i=l 

m. ESTIMATION OF PARAMETERS 
Maximum Likelihood 

The past performances of students may be used to obtain a maximum 
likelihood estimate of the p and q input vectors. 

Let a, . and b, . respectively be the number of incorrect and 



correct responses of students at level i who had given k correct 
responses at level i + 1, and define L(p,q) as the likelihood 
function of the vector (p,q). It follows from (3) that 



n m 



L(p.q)= n n (l-Pi)^'(l-qi+i)^ ^Ml-(l-Pi) (l-qi+/)^^^ . (9) 



Taking the partial derivatives of Ij(p,q) with respect to all pj and 
q. one obtains (10) and (11) setting them to zero yields (12) and (13) . 



m 



1=1 



•Pi l-d-PiXl-qi+i)* 



(10) 



4+1 



k= 0 



k-1 1 



1- 



"k+l l-(l-p.) (1-q.^^)* 



(11) 



m 
k=0 



m b^.(l-p.)(l-q.^^)^ 
k=0 l-(l-p.)(l-q.^j)^ 



(12) 



m 
k=0 



m kbj^.(l-pj) (l-qi+i) 

k=o i-(i-Pi)(i-qi+i)'' 



(13) 
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Note that in (12) and (13) the only unknown that parameter 
depends on is q^^j and vice versa. Thus the p and q vectors may 
be estimated by solving sets of two simultaneous equations in two 
unknowns. However, there is no analytical way of solving these 
equations in general for and ^i^i* Nevertheless, suppose 
one takes into account only the values of k with the highest number 
of observations. Denote these values by r and s. 

Then from (12) and (13) one obtains 



a . + a . = ■■ — - + (14) 

" " l-(l-Pi) (l-qj+l)' l-(l-Pi)(l-qi+/ 



rb^.(l-p.)(l-q.^/ ^ s bg.d-p.Xl-q.^^ )^ 

l-(l-Pi)(l-"qi+l)* i-(i-Pi)(i-qi+i)' 



r a .^-sa • = + • (15) 



Subtracting r times equation (14) from equation (15) yields 



(s-r)b^.(l-p,)(l-q.^^)^ 

(s-r) a . =. ; -— (16) 

" l-(l-Pi)(l-qi+/ 



which yields 

b .(l-p.)(l-q. i,)^ 

si^ ^l'^ ^1+1 



a . 

SI 



l-(l-Pi)(l-qi+l)' 



(17) 
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Si 



l-(l-Pi)(l-qi,l)' 



(18) 



From (17) and (18) one obtains 

(l-Pi)(l-qi+l)' = b = x; (19) 

ri ri 



SI SI 

where and are the proportion of unsuccessful performances 

at level i given r and s successful performances at level i + 1 
respectively. Solving these two equations for estimates of pj and 
q.^j one obtains 

s '^r 
_ s-r _ s-r 

p. = 1 -X^ X^ (21) 



s-r 



q. = l-(XyX^) . (22) 



Confidence Intervals 

Consider the two Bernoulli random variables, X and X , 

r s 

r s 
with parameters (l-Pj)(l-qj^j) and (l-Pj)(l-qj^j) respectively. 

From (19) and (20) it follows that X and X are the sample 

r s 

means of such random variables. Thus, it follows that for large sample 

r 

sizes I a 100 (1-p ) percent confidence interval for (l-Pi)(l-qj4.j) is given 
by: 

-17- 



L, = X ^ 
Ir r 

a . , - 
ri ri 



7 



(23) 



ri ri 



and for (1-Pi) (1-q.^^)^ by 



hs \ - / V ^s^l-X^) (25) 

T D . ^ 



V 



a 

SI SI 



SI SI 



where Z = Z^_p^2; that is PC^fZ^^p^^l = 1 - P /2 where Z 

is standard normal. * 

Since the probabilities that (L^^, L^^) amd {h^^,L^^) bracket 
(1-Pj) (l-qj+i) and (1-pj) (l-qj+i) are each 1-p and independent, the 
probability that both of these bounds hold is l-2p 1-2 p . 



*Larson (1966). 
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From this it follows that la^jL,^^ and l^^Jl^^^ form a 100 (l-2p) percent 



confidence interval for (l-q^) " . Similarly it follows that L^^/L^ 
and L|y/Ljg form 100 (1-p ) percent confidence interval for (1-p^)^'' . 



1 

These sets of bounds taken to the l/(s-r) jiower give 100(l-2p )°'^percent 

confidence intervals for (l-q^^^) and fll-p^) respectively. Consequently 

a lOO(l-Qf) percent confidence interval for p^ is given by 

1 

Pj = 1-(l|^/L^^)«-' (27) 

I 

^2 = l-<^lr/^L>'" «28) 
and for q.^j by 



^2 = 1 - <hs/^2r) 

where Lj^^, , Lj^., Lgj. are as defined in (15 - 18) and p = (l-a/2) 

Note that the above simplifies if s-r = 1 and that this is likely 
to happen as the two most likely choices for the number of successful 
performances to require at a given level are likely to be consecutive 
integers. 



6-r 
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A Linear Estimation Model 

In this model the student is given a questionnaire to 
determine his level of competence. Each question is scored one 
or zero, depending on whether the student answers the question 
right or wrong. The p^ »s and qJs for a student are each 
assumed to be linear combinations of the scores he receives 
on a question. Thusp 



p. = Z u. .w. 



(32) 



q. = E V..W. 

i ■ ' 

where w. is the score on question j for j > 1 and Wq = 1. 
This yields, for the vector pair (u,v), the likelihood function 

L(u.v) = n n_ (1- E U..W ) (1- Ev w ) n (1-(1- E u .w , ) 
i=l l|eX. j j '^^J i€X. i 

(33) 

(1 - ? v.^ljw/)! 



where w.^ is the score of student or trial i on question j and 
= j i : student responds incorrectly at level i | 
X. = { I: student responds correctly at level i j 



ERIC 



-20- 



• 



Note that in (33) k is actually a function of i. 

Taking partial derivatives of LCuyV) one obtains: 



2f;SH^ = L(u,v' 



1 



w..(l- 2v.,, .W.J 



(34) 



. V 1- Zu. .w.. 1-(1- Zu. .w.JU-Zv. .w.,) 



-kw. 



. Y 1- Sv.,, .w. , 
leXj i+lj j/ 

J 



.(l-Zu..w.^) ( l.Zv.,,.w./-^ 1 



(35) 



Setting both derivatives equal to zero, one obtains: 



« ^ Y 1- S U. .W.. 



(36) 



kw. 



kw.,(l..u,.w.^)(l.Zv,,,.w./-l 

J i i 



(37) 
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Since different values of i lead to different denominators in the 
terms of (36) and (37) , some simplification of these expressions in needed. 
Let: 

w.^jj^ = number of one scores for question j resulting in 
successful performances at level i following k 
successful performances at level i+l« 

^ijk ^ number of zero scores for question j resulting in 
successful performances at level i following k 
successful performances at level i+1, 

w*.jj^ = number of one scores for question j resulting in 
unsuccessful performances at level i following k 
successful performances at level i-i-l. 

^ijk ^ number of zero scores for question j resulting in 
unsuccessful performances at level i following k 
successful performances at level i+1. 

Suppose further that only question j is to be considered on the 
questionnaire. Then (36) and (37), when applied to n^y ^i+lj' ^iO' ^i+10 
become: 

m w',,, m w,,, (1-v, .,,-v. .-^) 

k=0 1-U..-U.0 k=0 l-(l-u..-u.o) (1-v.+i.-v.^^q) 



k=0 l-v.+y-v.^^Q k=0 1-(1-u..-u.q) (l-v.+i.-v.^^g)^ 



-22- 



• _ If 

m w«.., m wl.^ m w. ., (1-v. .,-v. 

k=0 l-u..-u.n k=0 1-u.n k=0 l-(l-u..-u.n)(l-v. .,.-v. .,-)^ 

ij lO lO ^ ij lO'^ 1+lj 1+10 



(40) 



^ ^ ^jk<^-Vio)^ 

k=0 l-(l-Uio)(l-v.^io)^ 



— k-I 

^ ^"^'ijk + ^ ^^ik ^ ^ ^^iik<^-\i-^o)<^-Vi-^i+io) 
^=0 i-^j-^o i^^o '''^=° i-(i-^j-^o''i-^i+i-^i+io)'' 



(41) 



+ E 



m t^^i^k<^-^0)<^-Vl0)^"^ 



k=o i-(i-Uio)a-vio)^ 



However, when (37) and (38) are multiplied by l-Uii-tt^o 

^^^y^^<'ome (12) and (13) with w«..j^. w..^. ^j-^^0'^i+lj"^VlO 
replacing a^^. b^^^, p^, and q^^^ respectively. 

Thus if one takes into account observations corresponding only 
to the two highest values of k, r and s, one obtains from (19- 22), 

s -r 



^2 V w. . + w.'. / \ w. . +wl . / 

\ ijr ijr / \ ijs ijs / 



(42) 
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1 



V. 



Vio 



= 1 - 



r 




(43) 



Subtracting (38) and (39) from (40) and (41) and then multiplying 
through by 1-u.Q and 1-v.^^Q yield (12) and (13) with w!.^, w^^, u.^, and 

^i+10 '^Pl^ci'^B \i» Pj. and q^^^ respectively. Thus, after 

considering only r and s as values of k, one obtains from (19>22) 
and (42) and (43). 




(44) 



1 



^i+10 



= 1 - 




(45) 




(46) 
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^i+lj 



w^. (w!. +w.. ) 
ijs' ijr ijr' 

w'.. (wl. + w.. ) 



1 

8-r 



w!. (w. . +w!. ) 

wl. (w.. + wl. ) 
Ijr^ IJS ijs' 



1 

s-r 



(47) 



In a similar manner one could also obtain lOO(l-cr) percent 

confidence intervals for u.., v..,., u.^, and v. .^^ by noting that 

ij i+lj' lO' 1+10 ^ ^ 

100(1- cy) percent confidence intervals for u^j + u^q and v^^^^ + v^^^q 

are also lOO(l-a) percent confidence intervals for u^j and v^^^^j s^d 

making the appropriate substitutions in (23-30). Note that the length 
/ of these confidence intervals tends to zero as the sample size for 
both k=r and k=8 tend to infinity and consequently the estimates in 
(44-47) are consistent. 

In (44-47) the weights are based on the assumption that all 
weights except for question j are zero. For the more general case 
where this is not assumed, it is suggested that the weights used be the 
average of the results in (44-47) taken over all j. That is: 

s 



S-r 




(48) 



w!. (w!. +w. . ) 
iis^ ijr ijr^ 

wl. (wl. +w.. ) 
ijr IJS IJS J 



1 

s-r 



(49) 
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n \ w.. +w!. / \ w.. +w!. / 

\ ijr ijr / \ ijs ijs / 



(50) 



s 



8-r 



'i+lj 



n 



1_ 
8-r 



n 



1 

8-r 



,i^0 (51) 



where n is tihe number of questions on the questionnaire. 

IV. EXAMPLE 

Consider the three level problem with the following data. 
(c^.Cg.Cj) = (6.7, 4.3. 1.9) 
(Pl'PZ'Ps^ = (-07. .23. .46) 

= (-6. .8) 
It then follows that Vj^(pj) = 6. 7/pj. 

-26- 
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For level 2, substitute the appropriate values for c^, Cj. p^, in 

(7) to obtain f(k.pj) = .995(.4)^ - 1.498 + 1.15/.4^'^ 

This yields f(l,p^) = .050, f(2,pj) = 1.536. Since the latter exceeds 
one, no more than one successful performance at level 2 will ever be 
required. This yields the following table for V.(p). 



i 


n(i) 


f 


0 


0 


.000 


1 


1 


.050 



Table 1: Output for level 2 

This table may be read as follows. For 0 < < .050, require no 
successful performances at level 2. For p2>.050, require one 
successful performance at level 2. 

For level 3, note from (8) that 

f3(k-l,l,p) = 1.9/ [VjCp, 1 -,77(.2)^"V V^Cp, l-.77(.2)^)] . 

In order to find f3(0,l,p), VjCp, l-.77(.2)®) = V2(p..23) and 

V2(p, 1-. T7(.2)) = V2^, .846) must be calculated by algorithm 2. For the 

calculation of V(p,.23), k(2) = 1 by table 1. Thus p^ = l-(.4) (.93) = .628 

andV(p,.23) =4.3/.23 + 6.7/.628 = 29.37. SimUarly, V(p, .846) = 15.75. 
Thus f3(0,l,p) = 1.9/(29.37 -15.75)= .140. SimUarly, V2(p,l-. 77(.2)^) = 

V2(p,.969) = 15.U and f(l,2,p) = 2.97. For k > 3, V2(p,l -. 77(.2)^'^) < 

V2(p,.969) = 15.U and V2(p, l-.77(.2)^ > V2(p,l) = 14.97. Thus 
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f(k-l,k,p) >4.3/(15.U - 14.97) = 30.71 > 1. Thus no more than 1 
successful performance at level 3 can be required, yielding the 
following table. 



i 


n(i) 


f 


0 


0 


.000 


1 


1 


.140 



Table 2: Output for level 3. 

Table 1 and 2 contain the information necessary to use algorithm 2 
to calculate V(. 07, .23, .46). From table 2, k(3) = 1 since .140 < .46 < 1. 
Thus p^ = 1 - . 77(.2) = .846. From table 1, k(2) = 1 since .140 < .628 < 1 

and ^ = 1- •93(.4) = .628. Thus one successful performance is required 

at each level and the expected time to complete the course is 
V(.07, .23. .46) = 6. 7/. 628 +4. 3/. 846 + 1.9/. 46 = 19.88. 



List of Symbols 



p = (pj, ...» pj^) state vector where p. is the probability 

student can perform at level i. 

probability student can perform at level 
i*l given that lie performs successfully 
at level i and could not previously per- 
form successfully at level i - 1. 

V(ir9 p) expected cost under policy ir when the 

system is in state p. 

V^(p) minimum cost for state p if we restrict 

instruction to levels I9 • • • » n. 

V^(p) same as (p) except that exactly k 

successful performances at level n are 
required. 

f(k, pj) the value of p^ such that V^(p) = vj"^(p) 

for fixed pj^. 

f^(j,k,p) the value of p^ such that vjcp) = V^(p), j < k. 

n n n*n 

a. . number of incorrect responses at level i 

following k correct responses at level i + 1. 

b. . number of correct responses at level i 

following k correct responses at level i + 1. 

L{p9q) the likelihood function of the vector ip^q). 

w. . . score on question j • 

u^j weight of question j upon p^. 

v. J weight of question j upon q. 

w.M * as defined in text, 
ijk 

w.M as defined in text, 
ijk 

w.M as defined in text, 

w. as defined in text. 

ERIC -29- 



References 



1. Atkinson, B.C. and Paulson, J. A., '*An Approach to the 
Psychology of Instruction,'* Psychological Bulletin. 1972,78, 49-61. 

2. Bellnian, R. , Dynamic Programming . Princeton, New Jersey: 
Princeton University Press, 1957. 

3. Bellman, R. and Dreyfus, S.E^, Applied Dynaimic Programming . 
Princeton: Princeton University Press, 1962 . 

4. Chant, V. G. , and Atkinson, R. C. , 'V Optimal Allocation of 
Instructional Effort to Interrelated Learning Strands, '* 
Journal of Mathematical Psychology. 1973, 10, 1-25. 

5. Howard, R. A. , Dynamic Programming and Markov Processes , 
New York; Wiley I960. 

6. Karush, W. and Dear, R.E. Optimal Stimulus Presentation 
Strategy for a Stimulus Sampling Model of Learning. 
Journal of Mathematical Psychology, 1966,3, 19-47. 

7. Larson, H. J., Introduction to Probability Theory and Statistical 
Inference. New York, John Wiley, 1969. 



-30- 



ONR DISTRIBUTION LIST 



Navy 

4 Dr. Marshall J. Farr, Director 
Personnel & Training Research 

Programs 
Office of Naval Research 
Arlington, VA 22217 

1 Director 

ONR Branch Office 
495 Summer Street 
Boston, MA 02210 
ATTN: C. M. Harsh 

1 Director 

ONR Branch Office 
1030 East Green Street 
Pasadena, CA 91101 
ATTN: E. E. Gloye 

1 Director 

ONR Branch Office 
536 South Clark Street 
Chicago, IL 60605 
ATTN: M. A. Bertin 

6 Director 

Naval Research Laboratory 
Code 2627 

Washington, DC 20390 

12 Defense Documentation Center 
Cameron Station, Building 5 
5010 Duke Street 
Alexandria, VA 22314 

1 Chairman 

Behavioral Science Department 
Naval Command & Management 

Division 
U.S. Naval Academy- 
Luce Hall 

Annapolis, MD 21402 



1 Chief of Naval Technical Training 
Naval Air Station Memphis (75) 
Millington, TN 38054 
ATTN: Dr. N. Kerr 

1 Chief of Naval Training 
Naval Air Station 
Pensacola, FL 32508 
ATTN: CAPT Bruce Stone, USN 

1 LCDR Charles J. Theisen, Jr., 
MSC, USN 4024 
Naval Air Development Center 
Warminster, PA 18974 

1 Commander 

Naval Air Reserve 
Naval Air Station 
Glenview, IL 60026 

1 Commander 

Naval Air Systems Command 
Department of the Navy 
AIR -4130 

Washington, DC 20360 

1 Mr. Lee Miller (AIR 413E) 
Naval Air Systems Command 
5600 Columbia Pike 
Falls Church, VA 22042 

1 Dr. Harold Booher 
NAVAIR 415C 

Naval Air Systems Command 
5600 Columbia Pike 
Falls Church, VA 22042 

1 CAPT John F. Riley. USN 
Commanding Officer 
U.S. Naval Amphibious School 
Coronado, CA 92155 



2. 



10 



Special Assistant for Manpower 
OASN (M&BA) 

The Pentagon, Room 4E794 
Washington, DC 20350 

Dr. Richard J. Neihaus 

Office of Civilian Manpower Management 

Code 06A 

Department of the Navy 
Washington, DC 20390 

CDR Richard L. Martin, USN 
COMFAIRMIRAMAR F-M 
NAS Miramar, CA 02145 

Research Director, Code 06 
Research and Evaluation Dept. 
U.S. Naval Examining Center 
Great Lakes, IL 60088 
ATTN: C. S. Winiewicz 

Chief 

Bureau of Medicine and Surgery 
Code 413 

Washington, DC 20372 

Program Coordinator 

Bureau of Medicine & Surgery 

(Code 71G) 
Department of the Navy 
Washington, DC 20372 

Commanding Officer 

Naval Medical Neuropsychiatric 

Research Unit 
San Diego, CA 92152 

Dr. John J. Collins 
Chief of Naval Operations (OP-987F) 
Department of the Navy 
Washington, DC 20350 

Technical Library (Per s -HE) 
Bureau of Naval Personnel 
Department of the Navy 
Washington, DC 20360 

Dr. James J. Regan 

Navy Personnel Research & 

Development Center 
San Diego, CA 92152 



Commanding Officer 
Navy Personnel Research 

and Development Center 
San Diego, CA 92152 

Supe rintendent 
Naval Postgraduate School 
Monterey, CA 92940 
ATTN: Library (Code 2124) 

Mr. George N. Graine 
Naval Ship Systems Command 
(SHIPS 03H) 

Department of the Navy 
Washington^ DC 20360 

Technical Library 
Naval Ship Systems Command 
National Center, Building 3 
Room 3S08 

Washington, DC 20360 

Commanding Officer 
Service School Command 
U.S. Naval Training Center 
San Diego, CA 92133 
ATTN: Code 303 

Chief of Naval Training Support 
Code N-21 
Building 45 
Naval Air Station 
Pensacola, FL 32508 

Dr. William L. Maloy 
Principal Civilian Advisor 
for Education & Training 
Naval Training Command 
Code 01 A 

Pensacola, FL 32508 

Dr. Hanss H. Wolff 
Technical Director (Code N-2) 
Naval Training Equipment 

Center 
Orlando, FL 32813 

Mr. Arnold Rubinstein 
Naval Material Command 

(NMAT.03424) 
Room 820, Crystal Plaza #6 
Washington, DC 20360 



ERIC 



Dr. H. Wallace Sinaiko 
c/o Office of Naval Research 
(Code 450) 

Psychological Sciences Division 
Arlington, VA 22217 

Dr. Martin F. Wiskoff 
Navy Personnel Research and 

Development Center 
San Diego, CA 92152 

Dr. John Ford, Jr. 
Navy Personnel Research and 

Development Center 
San Diego, CA 92152 

Technical Library 
Navy Personnel Research and 
Development Center 
San Diego, CA 92152 

Army 

Commandant 
U.S. Army Institute of 

Administration 
ATTN: EA 

Fcrt Benjamin Harrison, IN 46216 

Armed Forces Staff College 
Norfolk, VA 23511 
ATTN: Library 

Director of Research 

U.S. Army Armor Human Research Unit 
ATTN: Library 
Building 2422 Morade Street 
Fort Knox, KY 40121 

U.S. Army Research Institute for 
the Behavioral ScSocial Sciences 
1300 Wilson Boulevard 
Arlington, VA 22209 

Commanding Officer 
ATTN: LTC Montgomery 
USACDC- PASA 

Ft. Benjamin Harrison, IN 46249 

Dr. John L. Kobrick 
Military Stress Laboratory 
U.S. Army Research bistitute 
of Environmental Medicine 
Natick, MA 01760 



1 U.S« Army Research Institute 

Commonwealth Building, Room 239 
1300 Wilson Boulevard 
Arlington, VA 22209 
ATTN: Dr. R. Dusek 

1 Mr. Edmund F. Fuchs 

U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 

1 Chief, Unit Training St Educational 

Technology Systems 
U.S. Army Research Institute for 

the Behavioral and Social 

Sciences 
1300 Wilson Boulevard 
Arlington, VA 22209 

1 Commander 

U.S. Theater Army Support 

Command, Europe 
ATTN: Asst. DCSPER (Education) 
APO New York 09058 

1 Dr. Stanley L. Cohen 
Work Unit Area Leader 
Organizational Development Work 
Unit 

Army Research Institute for 

Behavioral and Social Science 
1300 Wilson Boulevard 
ArUngton, VA 22209 

1 Dr. Leon H. Nawrocki 

U.S. Army Research Institute 
Rosslyn Commonwealth Building 
1300 Wilson Boulevard 
Arlington, VA 22209 

Air Force 

1 Dr. Martin Rockway 

Technical Training Division 
Lowry Air Force Base 
Denver^ CO 80230 

1 Maj. P. J. DeLeo 

Instructional Technology Branch 
AF Human Resources Laboratory 
Lowry AFB, Co 80230 

1 Headquarters, U.S. Air Force 
Chief, Personnel Research & 

Analysis Division (AF/DPS Y) 
Washington, DC 20330 



-4- 



1 Pesearch and Analysis Division 
AF/DPXYR Koom4C200 
Washington, DC 20330 

1 AFHBL/AS (Dr. G.A. Eckstrand) 
Wright-Patterson AFB 
Ohio 45433 

1 AFHRL/IVID 

701 Prince Street 
Boom 200 

Alexandria, VA 22314 

1 AFOSR (NL) 

1400 Wilson Boulevard 
Arlington, VA 22209 

1 Commandant 

USAF School of Aerospace Medicine 
Aeromedical Library (SUL-4) 
Brooks AFB, TX 78235 

1 CAPT Jack Thorpe, USAF 
Department of Psychology 
Bowling Green State University 
Bowling Green, OH 43403 

1 Headquarters Electronic Systems 
Division 

ATTN: ^T. Sylvia R. Mayer/ MCIT 
LG Hanscom Field 
Bedfor, MA 01730 

I Lt. Col. Henry L. Taylor, USAF 
Military Assistant for Human 

Resources 
OAD (E&LS) ODDRg£ 
Pentagon, Room 3D129 
Washington, DC 20301 

Marine Corps 

1 COL George Caridakis 
Director 

Office of Manpower Utilization 
Headquarters, Marine Corps (AOIH) 
MCB 

Quantico, VA 22134 



1 Dr. A. L. Slafkosky 

Scientific Advisor (Code Ax) 
Commandant of the Marine Corps 
Washington, DC 20380 

1 Mr. E. A. Dover 

Manpower Measurement Unit 
(Code AOlM-2) 

Arlington Annex, Room 2413 
Arlington, VA 20370 

Coast Guard 

1 Mr. Joseph J. Cowan, Chief 

Psychological Research Branch(P-l) 
U.S. Coast Guard Headquarters 
400 Seventh Street, SW 
Washington, DC 20590 

Other POD 

1 Lt. Col. Austin W. Kibler, Director 
Human Resources Research Office 
Advanced Research Projects Agency 
1400 Wilson Boulevard 
Arlington, VA 22209 

1 Mr. Helga Yeich, Director 

Program Management, Defense 

Advanced Research Projects Agency 
1400 Wilson Boulevard 
Arlington, VA 22209 

1 Mr. William J. Stormer 
DOD Computer institute 
Washington Navy Yard 
Building 175 
Washington, DC 20374 

Other Government 

I Office of Computer Information 
Institute for Computer Sciences 

and Technology 
National Bureau of Standards 
Washington, DC 20234 



ERIC 



-5- 



1 Dr. Eric Mc Williams Program 1 
Manager 
Techndlogy and Systems, TIE 
National Science Foundation 
Washington, DC 20550 

Miscellaneous ^ 

1 Dr. Scarvia B. Anderson 
Educational Testing Service 
17 Executive, Park Drive, N.E. 
Atlanta, GA 30329 ^ 

1 Dr. Bichard C. Atkinson 
Stanford University 
Department of Psychology 
Stanford, CA 94305 ^ 

1 Mr. Edmund C. Berkeley 
Berkeley Enterpriser, Inc. 
815 Washington Street 
Newtonville, MA 02160 

1 Dr. Ronald P. Carver ^ 
American Institutes fdr Research 
8555 Sixteenth Street 
Silver Spring, MD 20910 

1 

1 Dr. Allan M. Collins 

Bolt Beranek and Newman 
50 Moulton Street 
Cambridge, MA 02138 

1 

1 Dr. Rene^ V. Dawis 
University of Minnesota 
Departmf'int of Psychology 
Minnfiapolis, MN 55455 

2 ERIC 1 
Processing and Reference Facility 
4833 Rugby Avenue 

Bethesda, MD 20014 

1 Dr. Victor Fields 

Department of Psychology 1 
Montgomery College 
Rockville, MD 20850 

1 Dr. Edwin A. Fleishman 

American Institutes for Research 
8555 Sixteenth Street 
Silver Spring, MD 20910 



Dr. Duncan N. Hansen 
Mempis State University 
Bureau of Educational Research 

and Services 
Memphis, TN 38152 

Dr. Robert Glaser, Director 
University of Pittsburgh 
Learning Research & Development 

Center 
Pittsburgh, PA 15213 

Dr. Henry J. Hamburger 
University of California 
School of Social Sciences 
Irvine, CA 92664 

Human Resources Research 

Organization 
Division #3 
P. O. Box 5787 

Presidio of Monterey, CA 93940 

Dr. Lawrence B. Johnson 
Lawrence Johnson and Associates, inc. 
200 S Street, N.W., Suite 502 
Washington, DC 20009 

Dr. David Kl^r 
Carnegie*Mellon University 
Graduate School of Industrial Admin. 
Pittsburgh, PA 15213 

Dr. Robert R. Mackie 
Human Factors Research, Inc. 
6780 Cortona Drive 
Santa Barbara Research Park 
Goleta, CA 93017 

Dr. Andrew R. Molnar 
Teclmological Innovations in 

Education 
National Science Foundation 
Washington, DC 20550 

Dr. Leo Munday 
Vice President 

American College Testing Progra;n 
P. O. Box 168 
Iowa City, lA 52240 



-6- 



1 Dr. Donald A. Norman 

University of California, San Diego 
Center for Human Information 

Processing 
La JoUa, CA 92037 

1 Dr. Diane M. Ramsey-Klee 

R-K Research ScSystem Design 
3947 Ridgemont Drive 
Malibu, CA 90265 

1 Dr. Leonard L. Rosenbaum 
Chairman 

Department of Psychology 
Montgomery College 
RockvUle, MD 20850 

1 Dr. George E. Rowland 

Rowland and Company, Inc. 
P. O. Box 61 
Haddonfield, NJ 08033 

1 Dr. Arthur I. Siegel 

Applied Psychological Services 
Science Center 
404 East Lancaster Avenue 
Wayne, PA 19087 

1 Mr. Dennis J. Sullivan 
725 Benson Way 
Thousand Oaks, CA 91360 

1 Dr. Benton J. Underwood 
Northwestern University 
Department of Psychology 
Evans ton, IL 60201 

1 Dr. David J. Weiss 

University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 

1 Dr. Anita West 

Denver Research Institute 
University of Denver 
Denver, CO 80210 

1 Dr. Kexmeth Wexler 

University of California 
School of Social Sciences 
Irvine, CA 92664 



1 Dr. John Annett 

The Open University 
Milton Keynes 
Buckinghamshire 
ENGLAND 

1 Dr. Milton Katz 
MITRE Corporation 
Westgate Research Center 
McLean, VA 22101 

1 Dr. Charles A. Ullmann 
Director 

Behavioral Sciences Studies 
Information Concepts, Inc. 
1701 N. Ft. Myer Drive 
Arlington, VA 22209 

1 Dr. Dexter Fletcher 
University of Illinois at 

Chicago Circle 
Department of Psychology 
Box 4348 

Chicago, IL 60680 

I Dr. Alfred F. Smode 
Staff Consultant 

Training Analysis and Evaluation 
Group 

Naval Training Equipment Center 
Code N-OOT 
Orlando, FL 32813 

1 U.S.C. Information Sciences Inst. 
Library 

4676 Admiralty Way 
Marina Del Key, CA 90291 

1 Mr. Rich Hale 

U.S.C. Information Sciences Inst. 
4676 Admiralty Way 
Marina Del Rey, CA 90291 

2 Jaclyn Caselli 
Director of Acquisitions 

ERIC Clearinghouse on Educational 

Media and Technology 
507 Stanford Center for R&D in 

Teaching 
Stanford University 
Stanford, CA 94305 



ERIC 



